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Abstract 

The  Military  Health  System's  (MHS)  benefit  program, 

TRICARE,  is  administered  through  a  direct  care  system  supported 
by  a  civilian  network  via  12  regional  Managed  Care  Support 
Contracts  (MCSCs) .  Lead  Agents,  whose  offices  serve  not  only  to 
oversee  MCSC  performance  but  also  to  manage  the  healthcare 
benefit  program  for  MHS  beneficiaries,  monitor  these  regional 
contracts.  As  such,  they  are  inundated  with  numerous  metrics, 
many  of  which  are  not  optimal  for  strategic  management.  This 
graduate  project  identifies  the  critical  performance  indicators 
for  the  Region  6  Lead  Agent  Office  (TRICARE  Southwest)  through  a 
comparative  analysis  of  similar  health  plans. 

Using  a  quantitative  decision-making  tool,  known  as  the 
JUDGE  (Judging  Utility:  a  Decision  Generator  and  Evaluator) 
Model,  the  performance  measurements  of  TRICARE  Southwest  were 
compared  with  those  of  TRICARE  Northwest  and  PacifiCare  of 
Texas.  Critical  performance  indicators  were  identified  for 
three  core  domains:  health  status,  operations/member  services, 
and  cost  accountability.  An  analysis  provides  the  executive 
staff  of  TRICARE  Southwest  recommendations  to  assist  in 
developing  an  optimal  performance  measurement  system  for  Region 
6.  The  critical  performance  indicators  identified  should  aid 
leaders  in  determining  the  "health"  of  the  health  plan. 
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Identifying  the  Critical  Performance  Indicators 
for  the  TRICARE  Lead  Agent  Office,  Region  6 

Introduction 

The  objective  of  this  Graduate  Management  Project  (GMP)  was 
to  conduct  an  analysis  of  the  performance  measurement  process  at 
the  office  of  the  Lead  Agent,  TRICARE  Southwest,  Department  of 
Defense  (DoD)  Health  Services  Region  (HSR)  6.  “The  Lead  Agent 
coordinates  the  delivery  of  health  care  and  integrates  the 
capabilities  of  the  military  treatment  facility  with  those  of 
the  contractor's  network"  (Johnson,  1996).  In  order  to 
accomplish  this,  the  Lead  Agent  staff  monitors  a  plethora  of 
metrics  in  overseeing  the  Managed  Care  Support  Contract  (MCSC) 
as  well  as  metrics  surrounding  the  direct  care  system.  For  the 
TRICARE  Program  to  be  effectively  and  efficiently  managed,  it 
was  imperative  that  the  proper  indicators  be  identified  and 
categorized  by  core  domain  for  not  only  operational  management 
but  also  benchmarking  capability.  The  Regional  Executive 
Council  (REC)  are  arguably  the  primary  stakeholders  for  the  Lead 
Agent,  and  their  interests  were  considered  in  the  performance 
measurement  process.  Furthermore,  future  Military  Health  System 
(MHS)  rightsizing  will  likely  involve  Lead  Agent  consolidations 
that  will  necessitate  a  streamlined  methodology  of  continuous 
quality  improvement  and  a  performance-based  management  system. 
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Conditions  Which  Prompted  the  Study 

In  March  1995,  the  MHS  began  implementation  of  DoD's 
managed  care  program,  TRICARE.  Twelve  HSRs  were  established  in 
the  Continental  United  States  (CONUS)  in  each  of  which  a  MCSC 
would  be  awarded.  These  MCSCs  would  supplement  the  care 
delivered  by  the  uniformed  services'  medical  treatment 
facilities  (MTFs)  in  providing  healthcare  to  beneficiaries  of 
the  Civilian  Health  and  Medical  Program  of  the  Uniformed 
Services  (CHAMPUS) .  As  of  June  1998,  all  DoD  CONUS  HSRs  are 
operating  a  MCSC  to  provide  care  to  their  beneficiaries.  Lead 
Agents  are  responsible  for  MHS  oversight  of  the  MCSCs  and  have  a 
variety  of  associated  responsibilities.  In  November  1995,  the 
Office  of  the  Assistant  Secretary  of  Defense  (Health  Affairs) 
(OASD) (HA)  provided  the  Lead  Agents,  through  the  Service  Surgeon 
Generals,  guidelines  "designed  to  clarify  Lead  Agents'  roles 
without  being  prescriptive"  (U.S.  Department  of  Defense,  1995) . 
Included  in  these  guidelines  was  the  charge  to  "participate  in 
MHS  metric  standardization  of  data  to  be  reported/used"  (U.S. 
Department  of  Defense) . 

One  month  later,  senior  staff  members  of  OASD (HA) 
formulated  a  series  of  performance  metrics  that  ultimately  led 
to  the  MHS  Performance  Report  Card  initiative.  In  September 
1998,  the  MHS  Performance  Report  Card  initiative  was  replaced  by 
the  TRICARE  Operational  Performance  Statement  (TOPS) .  TOPS  is 
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"both  a  statement  and  an  evaluation  tool  used  by  the  leadership 
and  staff  of  the  TRICARE  Management  Activity  (TMA)  and  the  OASD- 
HA  to  get  a  snapshot  of  the  performance  of  the  MHS  at  quarterly 
intervals"  (TRICARE  Management  Activity,  1998) .  The  TOPS  is 
not,  however,  designed  to  provide  the  Lead  Agent  with  management 
feedback  for  regional  improvement  activities  (Constantian,  A., 
personal  communication,  October  19,1998). 

Each  MCSC  specifies  certain  management  reports  to  be 
provided  to  the  government  by  the  contractor,  but  these  metrics 
are  not  standardized  among  the  Regions.  With  TRICARE  fully 
implemented,  it  is  critical  that  the  Lead  Agents  establish  and 
monitor  those  performance  indicators  that  facilitate  effective 
decision-making.  A  literature  review  coupled  with  an  analysis 
of  the  management  indicators  available  to  the  Region  6  Lead 
Agent  and  those  used  by  comparative  organizations  will  help 
identify  the  critical  performance  measurements  needed  for 
optimal  health  plan  oversight.  The  recommendations  could 
potentially  serve  as  a  template  for  those  TRICARE  Lead  Agents 
just  beginning  managed  healthcare  delivery  or  for  consolidated 


Lead  Agents. 
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Statement  of  the  Problem 

Lead  Agent  offices  are  inundated  with  a  variety  of  metrics 
generated  both  internally  and  externally.  As  they  face  reduced 
resources,  those  in  leadership  positions  must  be  armed  with  a 
clear  understanding  of  the  critical  indicators  to  be  monitored. 
The  question  posed  is:  what  performance  measurements  should  be 
generated  and  monitored  for  the  effective  management  of  TRICARE 
in  Region  6?  This  involves  identifying  the  performance 
indicator  candidates,  categorizing  them  based  upon  Lead  Agent 
core  domains,  verifying  the  quality,  accessibility,  and 
timeliness  of  the  data,  determining  the  indicators'  strategic 
link,  assessing  the  value  of  the  indicators  to  stakeholders,  and 
specifying  the  recommendations. 

Literature  Review 

Performance  measurement  in  the  healthcare  setting  refers  to 
the  use  of  process  measures  and  outcomes  to  understand 
organizational  performance  and  to  affect  positive  change  to 
improve  care  (Nadzam  &  Nelson,  1997) .  While  the  performance  of 
hospitals  has  been  measured  for  some  time  by  various 
organizations,  health  plan  accountability  in  the  managed  care 
setting  is  much  newer  (Spoeri  t  Ullman,  1997) .  A  common  term 
used  to  denote  published  summaries  of  plan  performance  for  a 
specified  period  of  time  is  “report  card." 


A  report  card  can 
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provide  "information  about  clinical  outcomes,  cost- 
effectiveness,  and  organizational  performance  in  an  era  when 
healthcare  organizations  are  competing  for  marketshare  and 
consumers  are  demanding  to  be  informed  about  their  healthcare 
providers"  (Slovensky,  Fottler,  &  Houser,  1998) . 

A  large  number  and  variety  of  organizations  produce  health 
plan  report  cards  for  three  principal  user  groups:  purchasers, 
consumers,  and  health  plans  (Allen  &  Rogers,  1996).  Employers 
desire  health  plan  quality  information  to  assess  the  value  that 
they  are  receiving  for  their  expenditures  on  employee  and 
retiree  healthcare  benefits.  Report  cards  assist  consumers  in 
evaluating  the  various  available  health  insurance  options. 
Finally,  health  plans  can  use  the  information  to  assist  in 
continuous  quality  improvement  (CQI)  efforts  (Scanlon,  Chernew, 
Sheffler,  &  Fendrick,  1998). 

While  in  many  cases  the  report  card  initiatives  in  the 
commercial  sector  have  been  voluntary  responses  to  a  perceived 
public  or  employer  desire  for  comparative  data,  they  are 
increasingly  accomplished  with  an  underlying  goal  of 
accreditation.  The  National  Committee  for  Quality  Assurance 
(NCQA)  and  the  Joint  Commission  on  Accreditation  of  Healthcare 
Organizations  (JCAHO)  are  likely  the  most  reputable  accrediting 
organizations  with  national  performance  measurement  systems. 

The  NCQA's  Health  Employer  Data  and  Information  Set  (HEDIS) 
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originated  in  1989  by  a  coalition  of  employers  and  insurers 
seeking  to  establish  standardized  measurements  for  health  plan 
comparison.  JCAHO's  initiative.  Oryx,  was  introduced  in 
February  1997  and  allows  organizations  to  build  their  own  report 
cards  (O'Malley,  1997) . 

Regardless  of  who  develops  them,  report  cards  typically 
compare  plans  on  one  or  more  of  the  following  types  of 
information:  "access  to  care,  use  of  services,  clinical 
indicators  of  quality  such  as  immunization  rates  and  surgical 
outcomes,  measures  related  to  finances  and  administrative 
efficiency,  information  on  costs,  and  a  variety  of  patient- 
derived  measured  including  complaint  data  and  results  from 
member  satisfaction  surveys"  (McGee  &  Knutson,  1994)  .  The 
balance  between  administrative  and  clinical  information  included 
largely  depends  upon  the  intended  users  of  the  report  card.  As 
a  result,  those  who  develop  report  cards  should  employ  a  sound 
technical  approach  to  measurement  (McGee  &  Knutson) . 

Issues  to  consider  when  establishing  the  performance 
measurement  system  are  "examining  the  appropriateness  and 
feasibility  of  measures  being  considered  for  inclusion  in  report 
cards,  standardizing  definitions  and  formulas,  determining 
sampling  strategies  and  risk  adjustments  to  be  made,  identifying 
limitations  in  the  data,  and  discussing  ways  to  avoid 
misinterpretation"  (McGee  &  Knutson) .  Furthermore,  health  plans 
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must  avoid  focusing  their  resources  solely  on  those  items 
measured  at  the  expense  of  other  quality  improvement  activities 
that  might  be  of  greater  importance  to  their  own  system  and 
population  needs .  This  necessitates  that  management  personnel 
maintain  a  "big  picture"  view  of  the  corporate  environment 
(McGee  &  Knutson) . 

The  literature  identifies  several  other  factors  to  consider 
as  well  when  establishing  performance  measurement  activities. 

In  their  article  "Developing  a  Corporate -Level  Performance 
Assessment  System,"  Laffel,  Thompson,  &  Sparer  (1995)  describe  a 
medical  center's  experience  in  establishing  a  report  card.  They 
found  a  major  issue  to  be  considered  is  the  tradeoff  between 
brevity  and  comprehensiveness.  This  is  certainly  germane  for 
health  plans  also  as  they  evaluate  the  extensiveness  of  their 
metrics.  Alsever,  Ritchey,  &  Lima  (1995)  found  that  for  the 
Sisters  of  Charity  Healthcare  System  to  demonstrate  value  in  a 
report  card  system,  clear  goals  had  to  first  be  identified  while 
recognizing  that  "the  automation  of  data  collection  and  the 
ability  to  quickly  analyze  and  trend  data  are  critical  to 
quality  improvement . " 

"Health  plans  increasingly  realize  that  quality  management, 
including  performance  reporting,  is  critical  to  their 
competitive  success"  (Heinen,  Peterson,  Pion,  &  Leatherman, 

1993) .  While  Jackson  &  Kroenke  (1997)  indicate  that  competition 
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is  a  driving  force  only  for  non-Federal  health  care  systems, 
recent  governmental  activities  indicate  otherwise.  Increased 
Congressional  interest  in  the  MHS  has  resulted  in  not  only 
numerous  General  Accounting  Office  studies  on  cost-effectiveness 
as  compared  to  civilian  health  systems,  but  also  the  initiation 
of  demonstration  projects  such  as  TRICARE  Senior  Prime  and  the 
Federal  Employees  Health  Benefits  Plan  65  (FEHBP-65) .  Without  a 
doubt,  leaders  of  the  MHS'  managed  care  program,  TRICARE,  must 
recognize  the  necessity  of  performance  measurement  processes . 

Fundamental  to  their  success  will  be  the  regional  Lead 
Agent  staff's  ability  to  benchmark  their  performance  indicators. 
Benchmarking  can  be  defined  as  "the  continuous  process  of 
measuring  products,  services,  and  practices  against  the  toughest 
competitors  or  those  companies  recognized  as  industry  leaders" 
(Patrick  &  Alba,  1994) .  For  Lead  Agents,  these  comparisons 
should  not  only  include  the  federal  healthcare  industry  leaders 
but  also  civilian  health  plans  that  are  increasingly  becoming 
competitors  of  the  MHS.  Numerous  studies  (Gordan,  1996;  Jordan, 
Straus,  &  Bailit,  1995;  Joshi,  1994;  Patrick  &  Alba,  1994) 
highlight  the  benefits  of  benchmarking  in  healthcare  and  provide 
lessons  learned  from  previous  initiatives. 

"As  the  Department  of  Defense  transitions  to  a  capitated 
health  care  system  modeled  after  managed  care  organizations, 
assessment  and  accountability  of  system  effects  on  the  quality 
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of  health  care  delivery  are  critical"  (Jackson  &  Kroenke,  1997)  . 
This  challenge  is  exacerbated  by  the  primary  readiness  mission 
of  the  MHS  (Paul,  1997) .  In  order  to  appropriately  define  the 
critical  performance  indicators,  the  Lead  Agent  executive  staff 
“must  view  the  reporting  process  as  important  from  a  strategic 
perspective,  and  establish  processes  and  allocate  resources  to 
produce  a  report  in  an  efficient  and  cost-effective  manner  that 
will  meet  the  information  needs  of  key  stakeholders"  (Slovensky 
et  al,  1998) . 

Purpose 

The  purpose  of  this  study  is  to  identify  the  critical 
performance  metrics  to  support  executive  decisions  in  the 
management  of  TRICARE  Southwest.  As  suggested  in  the  literature 
review,  this  will  involve:  assessing  the  performance  indicators 
currently  in  use,  exploring  the  quality  improvement  initiatives 
of  other  federal  as  well  as  civilian  health  plans,  determining 
the  interests  of  the  executive  management  and  key  stakeholders, 
verifying  the  capabilities  for  collecting  quality,  accessible, 
and  timely  data,  and  recommending  a  performance  measurement 
system  for  establishment  at  the  Lead  Agent  Office,  DoD  HSR  6. 
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Methods  and  Procedures 

The  strategy  for  accomplishing  this  GMP  involved 
naturalistic  inquiry,  qualitative  data  collection,  and  content 
analysis  using  a  decision-making  tool  (Patton,  1989) .  This 
approach  provided  the  best  avenue  for  ensuring  a  flexible 
process  for  the  identification  of  a  value-added  performance 
measurement  system.  Naturalistic  inqpairy  involved  the 
observation  of  internal  and  external  Lead  Agent  Office 
operations  (e.g.  the  Regional  Executive  Council  meetings,  the 
MCS  Contractor's  Program  Management  Reviews,  and  the  Lead  Agent 
staff's  strategic  planning  sessions)  and  the  investigation  of 
two  other  managed  care  organizations'  performance  measurement 
activities.  The  qualitative  data  collected  would  then  be 
systematically  compared,  using  a  decision-making  tool,  to 
determine  the  optimal  performance  indicators. 

Data  Collection 

The  existing  management  metrics  for  TRICARE  Southwest  were 
obtained  through  a  variety  of  sources.  This  included 
interactions  with  Lead  Agent  staff  members,  involvement  with  the 
contractor's  Program  Management  Review  meetings,  and 
Participation  in  the  Lead  Agent  staff's  strategic  planning 
sessions.  This  effort  was  timely  as  the  Office  of  the  Region  6 
Lead  Agent  underwent  reorganization  in  Winter  1998-99,  and  new 
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leadership  sought  opportunities  for  improvement  under  a  - 
visionairy  management  framework . 

There  were  numerous  metrics  gathered,  and  all  were 
categorized  by  description,  point  of  contact,  source,  and 
frequency  of  monitoring.  The  Contract  Data  Requirements 
Listings  (CDRLs)  specify  reports  and  metrics  that  the  Managed 
Care  Support  Contractor  is  required  to  submit  to  the  Government. 
Of  the  hundreds  of  CDRLs  originally  identified  for  contract 
monitoring,  the  Lead  Agent  staff  has  narrowed  the  necessary 
jfegional  submissions  to  63,  which  were  included  in  the  study. 
Over  180  metrics  reviewed  through  the  Program  Management  Review 
were  also  catalogued  in  the  list  of  existing  measurements. 
Finally,  the  metrics  established  for  the  TRICARE  Senior  Prime 
Program  were  included.  These  metrics  were  identified  by  the 
TRICARE  Senior  Prime  Quality  Council  as  the  performance 
indicators  for  their  health  plan.  The  listing  of  existing 
metrics  for  TRICARE  Southwest  is  identified  in  Appendix  A. 

In  order  to  provide  an  appropriate  comparison,  the 
performance  measurement  activities  of  the  TRICARE  Northwest  Lead 
Agent  Office  (HSR  11)  and  a  local . civilian  health  plan, 
PacifiCare  of  Texas,  were  also  selected  for  assessment.  Region 
11  provided  an  example  of  another  DoD  Lead  Agent's  approach  with 
the  same  MCS  contractor.  Furthermore,  TRICARE  Northwest  was  a 
more  mature  region  and  had  more  experiences  to  draw  upon.  Their 
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metrics  are  listed  in  Appendix  B  and  include  the  metrics 
supporting  their  "Border  to  Border  Goals  and  Objectives"  and 
selected  ones  from  the  Region  11  Program  Management  Review. 
PacifiCare  served  as  an  example  of  another  plan's  activities 
within  the  same  geographical  region.  As  a  non-governmental 
health  plan,  it  also  facilitated  some  other  unique  comparisons. 
PacifiCare 's  metrics  are  included  in  Appendix  C  and  include 
metrics  that  reflect  their  for-profit  status. 

Once  the  existing  metrics  for  TRICARE  Southwest,  TRICARE 
Northwest,  and  PacifiCare  were  collected,  the  second  step  was  to 
determine  their  candidacy  as  critical  performance  indicators  for 
TRICARE  Southwest.  All  metrics  were  considered  as  candidates 
unless  one  of  the  following  conditions  applied:  (1)  the 
identified  measurement  was  not  actually  a  metric  (e.g.  a 
narrative  report  such  as  a  network  provider  listing) ;  (2)  the 
measurement  was  an  MTF,  FHFS,  or  external-organization  specific 
metric  (e.g.  an  operating  profit  margin) ;  or  (3)  the  measurement 
was  not  a  metric  for  executive  level  management  consideration 
(e.g.  a  total  number  of  surveys  processed) .  When  there  was  any 
uncertainty,  the  point  of  contact  for  the  metric  was  queried 
regarding  the  potential  candidacy.  These  determinations  are 
included  in  the  respective  metrics'  listings  in  appendices  A,  B, 


and  C. 
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The  next  step  in  the  critical  performance  indicator 
identification  process  was  to  determine  the  core  domains  or 
performance  drivers  for  TRICARE  Southwest  and  to  verify  primary 
stakeholder  acceptance.  The  Executive  Staff,  during  a  strategic 
management  off-site  meeting,  identified  the  organization's  3 
core  domains:  health  status,  operations/member  services,  and 
cost  accountability.  Subsequently,  the  Executive  Staff  drafted 
a  goal  for  each  domain.  The  3  domains  and  their  goals  were 
clearly  supported  by  the  agenda  of  a  primary  stakeholder  in  the 
performance  measurement  process,  the  Regional  Executive  Council 
(REC) . 

The  REC's  membership  is  comprised  of  the  MTF  Commanders 
within  the  Southwest  Region,  to  include  13  Air  Force  facilities, 
4  Army  facilities,  and  1  Navy  facility.  The  REC  meets  on  a 
quarterly  basis  and  discusses  issues  facing  the  TRICARE  program. 
As  a  part  of  this  process,  they  define  the  Critical  Initiatives 
for  the  Lead  Agent  staff  to  target  as  opportunities  for  - 
performance  improvement  activities.  The  most  recent  REC 
meetings  were  held  in  November  1998  and  February  1999,  during 
which  their  discussions  and  decisions  supported  the  core  domains 
and  goals  for  regional  performance  measurement .  Members  of  the 
REC  agreed  that  the  role  of  the  Lead  Agent's  Office  is  not 
solely  to  monitor  contract  compliance,  but  to  serve  as  a  partner 
with  the  Managed  Care  Support  Contractor  in  an  effort  to  build 
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and  maintain  the  best  healthcare  system  possible.  As  such, 
these  key  stakeholders  expect  the  Lead  Agent  staff  to  identify 
and  track  the  fundamental  measurements  for  operating  a 
successful,  regional  healthcare  delivery  system. 

In  this  effort,  the  next  step  was  to  organize  the  potential 
critical  performance  indicators  by  the  3  core  domains  and 
specific  metric  types.  This  step  is  important  to  ensure  that 
the  final  metrics  identified,  following  the  content  analysis, 
can  assist  the  Executive  Staff  in  determining,  for  each  goal, 
potential  objectives  that  are  definable  and  measurable.  This 
also  allowed  the  combination  of  metrics  from  the  3 
organizations,  to  eliminate  redundancy  and  produce  a 
consolidated  list  of  critical  performance  indicator  candidates. 


Table  1  depicts  the  14  resulting  categories  for  content 
analysis . 

Table  1 


Categories  for  Content  Analysis 


Operations/ 


Health  Status 
(Appendix  D) 

Utilization 
Management 
Quality  Management 
Case  Management 
Health  Care 
Information  Lines 
Clinical  Indicators 


Member  Services 
(Appendix  E) 


Provider  Services 
Customer  Service 
Customer  Satisfaction 


Enrollment 
Access  to  Care 


Cost 

Accountability 
(Appendix  F) 

Enrollment  Based 
Capitation 
Resource  Sharing 
claims  Processing 


Pharmaceuticals 
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Content  Analysis 

The  next  stage  in  the  process  of  identifying  the  critical 
performance  indicators  for  TRICARE  Southwest  involved  the  use  of 
a  quantitative  decision-making  tool,  known  as  the  JUDGE  (Judging 
Utility:  a  Decision  Generator  and  Evaluator)  Model.  The  JUDGE 
model  provides  an  analytical  method  of  making  a  decision  and 
provides  a  mathematical  best  choice.  However,  the  user  of  the 
model  may  have  to  consider  additional  factors  before  making  the 
final  decision.  Using  the  JUDGE  Model  allowed  the  potential 
performance  indicators  identified  for  Region  6  to  be  compared 
with  those  of  TRICARE  Northwest  and  PacifiCare  of  Texas,  by  core 
domain  and  metric  type . 

The  first  step  of  the  model  was  to  specify  the  attributes  of 
competing  alternatives.  In  this  case,  that  involved  specifying 
the  important  attributes  of  a  good  metric.  Seven  attributes 
were  used  to  describe  the  potential  metrics:  data  quality,  data 
accessibility,  data  timeliness,  data  actionable,  strategic  link, 
external  benchmark,  and  stakeholder  value. 

The  attributes  used  were  identified  in  the  literature 
(Gundacker,  1998;  National  Performance  Review,  1997;  Roadman, 
1999)  as  fundamental  characteristics  of  good  performance 
measurements.  Data  quality,  listed  first,  refers  to  the 
accuracy  of  the  metric's  source  data.  Secondly,  the  data's 
accessibility  indicates  the  ease  or  difficulty  in  obtaining  the 
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metric's  source  data.  The  currency  of  the  metric's  source  data 
is  assessed  by  the  third  attribute,  data  timeliness.  The  fourth 
attribute,  termed  data  actionable,  refers  to  the  meaningfulness 
of  the  metric  and  the  degree  to  which  it  facilitates  executive 
level  decision-making.  The  fifth  attribute  identified  as 
strategic  link  indicates  whether  or  not  the  metric  is  aligned 
with  the  TRICARE  Southwest's  mission,  vision,  and  goals.  Sixth, 
external  benchmark  evaluates  the  proliferation  of  the  metric 
among  the  3  organizations.  Finally,  the  seventh  attribute, 
stakeholder  value,  compares  the  metric  to  the  critical 
initiatives  and  key  interests  of  TRICARE  Southwest's  primary 
stakeholder,  the  REC. 

The  second  step  of  the  model  was  to  rate  the  attributes. 
Each  attribute  was  rated  on  a  9 -point  scale.  The  bipolar 
relative  rating  scale  is  anchored  at  each  point,  with  the 
measurement  of  1  representing  extremely  undesirable  and  9 
^®P^®senting  extremely  desirable.  While  none  of  the  attributes 
are  undesirable,  their  significance  in  the  identification  of  the 
critical  performance  indicators  is  scored  relative  to  one 
another . 

Ratings  were  then  recoded  and  rescaled  (step  three)  so  that 
a  neutral  point  of  zero  could  be  obtained.  This  was 
accomplished  by  subtracting  5  from  each  of  the  ratings 
previously  assigned  on  the  9-point  scale.  The  ratings  were  then 
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rescaled  by  multiplying  the  recoded  ratings  by  a  scaling  factor 
of  5.56.  This  factor  is  used  so  that  the  rescaled  ratings  sum 
to  100 .  The  same  initial  and  recoded  ratings  were  included  in 
all  14  JUDGE  Model  calculations  that  are  found  in  Appendix  G. 

Step  four  was  to  identify  the  alternatives.  There  are  a 
multitude  of  metrics  in  the  healthcare  industry.  Again,  the 
alternatives  were  limited  to  those  critical  performance 
indicator  candidates  that  are:  (1)  already  used  at  TRICARE 
Southwest,  (2)  used  at  TRICARE  Northwest,  and  (3)  used  at 
PacifiCare  of  Texas.  Writing  the  alternative  equations  in 
linear  form  produces  the  following: 

■  Alternative  A  (Current  Metric  at  TRICARE  SW) : 

y‘^’=iWiV‘^^+iW2V‘^’+ — iWioV‘^°’ 

■  Alternative  B  (Metric  at  TRICARE  NW) : 

Y‘2)  =2WiV‘^’  +2W2V‘^’  + - 2WioV‘^°’ 

■  Alternative  C  (Metric  at  PacifiCare  of  TX) : 

Y‘^’=3WiV‘^’+3W2V‘^’+ - 3WioV‘^°’ 

where  Y  is  the  alternative,  w  is  the  attribute's  utility,  and  V 
is  the  rescaled  rating. 

Step  six  was  to  judge  each  attribute's  utility  in  relation 
to  each  attribute.  Table  2  indicates  the  method  used  in 
determining  the  utility  of  each  attribute: 
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Table  2 

Methods  Used  to  Determine  Attribute  Utility 


Method  Used  For 

Attribute  Attribute  Determining 

Nuaaber _ Description  Utility  Source 


1 

Data  Quality 

Data  Review 

LA  Info  Mgt 

2 

Data  Accessibility 

Data  Review 

LA  Info  Mgt 

3 

Data  Timeliness 

Data  Review 

LA  Info  Mgt 

4 

Data  Actionable 

Data  R^eview 

Resident 

5 

Strategic  Link 

Exec.  Interaction 

Resident 

6 

External  Benchmark 

Data  Review 

Resident 

7 

Stakeholder  Value 

REC  Review 

The  utility  for  attributes  1-3  was  based  on  the  responses  of  the 
Lead  Agent's  Chief  of  Data  Quality  and  Analysis,  Captain  Bama 
Lambert,  as  she  investigated  the  metrics.  For  attributes  4-7, 
the  utility  of  each  attribute  was  judged  using  a  subjective, 
"best  estimate"  method  based  upon  the  Resident's  interactions 
with  the  Executive  Management  Team,  Lead  Agent  Staff  Members, 
the  Regional  Executive  Council,  and  representatives  of  the  two 
external  organizations.  This  involved  assessing  the  importance 
of  the  metric  to  the  organization  based  upon  its  need  in 
executive  level  decision-making  activities. 

Step  seven  was  a  comparison  of  alternative  decision  indices. 
Calculating  a  weighted  composite  score  for  the  alternatives  for 
each  attribute  allowed  a  comparison  of  alternatives.  The 
weighted  composite  score  was  calculated  by  multiplying  the 
valence  for  each  attribute  by  the  weights  for  each  alternative. 
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Evaluation  of  the  decision  components  was  the  eighth  step. 
Appendix  G  shows  the  final  attribute  scores  for  each 
alternative.  After  reviewing  these  attributes  it  was  decided 
that  no  adjustment  to  ratings  or  weighted  values  would  be 
necessary.  The  final  step  of  the  model  is  to  determine  the 
conclusion.  Based  upon  the  JUDGE  model  analyses,  scores  were 
calculated  for  each  metric  type  and  are  also  included  in 
Appendix  G. 

The  Results 

Health  Status 

JUDGE  Model  calculations  under  the  health  status  core 
domain  were  accomplished  for  utilization  management,  quality 
management,  case  management,  clinical  indicators,  and  healthcare 
information  lines.  For  all  five  metric  types,  TRICARE 
Southwest's  method  scored  highest.  With  the  exception  of 
healthcare  information  lines  (which  PacifiCare  data  did  not 
include),  PacifiCare' s  method  scored  second  for  each  of  the 
metric  types.  Several  explanations  exist  for  these  findings. 

The  success  of  the  TRICARE  Southwest  performance  indicators 
for  health  status  is  largely  due  to  the  metrics  that  were 
included  from  the  TRICARE  Senior  Prime  demonstration.  The 
utilization  management,  quality  management,  and  case  management 
metrics  that  are  being  used  with  the  Senior  Prime  population 
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should  also  be  considered  for  use  with  the  Prime  population. 

This  would  mirror  the  practice  used  by  PacifiCare,  whereby 
metrics  examined  for  the  senior  population  are  also  measured  for 
their  commercial  product.  Furthermore,  it  should  be  noted  that 
the  clinical  indicators  of  PacifiCare  outscored  those  of  TRICARE 
Southwest  on  the  attributes  of  data  actionable,  strategic  link, 
and  stakeholder  value.  However,  the  pervasive  lack  of  quality, 
accessible,  and  timely  MHS  clinical  data  from  the  TRICARE 
Southwest  perspective  largely  prohibits  PacifiCare 's  clinical 
indictors  from  usage  at  the  Region  6  Lead  Agent. 

While  TRICARE  Northwest  certainly  has  metrics  measuring 
health  status,  their  minimal  representation  in  the  metric  types 
examined  resulted  in  lower  scores.  Since  TRICARE  Northwest  also 
includes  a  TRICARE  Senior  Prime  demonstration,  it  is  likely  that 
they  actually  have  more  clinical  metrics  than  were  included  in 
this  study.  Finally,  their  measurement  for  the  healthcare 
information  line  is  identical  to  that  used  by  TRICARE  Southwest, 
since  the  metric  is  a  CDRL  of  the  MCSC,  Foundation  Health 
Federal  Services . 

Operations /Member  Services 

JUDGE  Model  calculations  under  the  operations /member 
services  core  domain  were  accomplished  for  provider  services, 
customer  service,  customer  satisfaction,  enrollment,  and  access 
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to  care.  The  results  for  this  core  domain  were  mixed  across  the 
three  organizations,  and  there  were  some  noteworthy  trends . 

TRICARE  Southwest  scored  highest  for  both  provider  services 
and  customer  service.  The  importance  of  examining  network 
adeguacy  coupled  with  issues  such  as  provider  satisfaction 
highlighted  Region  6's  efforts.  Measuring  customer  seirvice 
metrics  are  included  in  the  quarterly  performance  management 
review  of  the  MCSC,  and  emphasize  compliance  with  contractual 
standards.  PacifiCare,  however,  best  measured  customer 
satisfaction,  through  a  process  examining  member  complaints. 

TRICARE  Northwest  scored  highest  for  measuring  enrollment 
as  well  as  access  to  care.  While  the  comprehensive  list  of 
metrics  surrounding  enrollment  is  similar  between  Regions  11  and 
6,  TRICARE  Northwest  has  carefully  extracted  those  specific 
performance  measurements  that  are  worthy  of  executive  oversight. 
This  resulted  in  higher  weighted  composite  scores  for  their 
enrollment  indicators.  In  measuring  access  to  care,  TRICARE 
Northwest  excels  by  examining  not  only  the  civilian  network 
compliance  with  access  standards,  but  also  the  performance  of 
the  regional  MTFs  or  the  direct  care  system.  This  provides 
Region  11 's  executive  team  with  a  snapshot  of  the  entire 
region's  performance. 
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Cost  Accountability 

JUDGE  Model  calculations  under  the  cost  accountability  core 
domain  were  accomplished  for  enrollment  based  capitation, 
resource  sharing,  claims  processing,  and  pharmaceuticals. 

TRICARE  Northwest  performance  indicators  scored  a  perfect  100 
for  three  of  the  metric  types:  enrollment  based  capitation, 
resource  sharing,  and  pharmaceuticals. 

Enrollment  based  capitation  (EEC)  and  resource  sharing  are 
reflective  of  relatively  new  business  practices  for  the  MHS.  In 
fact,  EEC  has  not  yet  been  implemented  as  a  funding  methodology 
for  the  MHS.  Anticipating  this  future  budgeting  methodology, 
TRICARE  Northwest  monitors  performance  indicators  both  inter- 
and  intra-  regionally  to  assess  their  health  plan' s  financial 
condition.  While  TRICARE  Southwest  is  aware  of  resource  sharing 
target  savings  versus  actual  savings,  TRICARE  Northwest  monitors 
this  regularly  as  a  formal  objective.  Moreover,  their  executive 
staff  also  monitors  regional  pharmaceutical  costs  through 
inventory  levels,  utilization  summary,  and  other  health 
insurance  off-sets.  Data  from  PacifiCare  did  not  indicate  any 
monitoring  activity  surrounding  pharmaceuticals. 

Claims  processing  metrics  for  TRICARE  Northwest  scored  just 
higher  than  those  of  TRICARE  Southwest .  Claims  processing  has 
increasingly  become  an  area  of  focus  for  the  MHS,  and  as  a 
result,  are  receiving  additional  attention  by  performance 
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measurement  activities.  Areas  of  emphasis  include  claims' 
inventory  levels,  electronic  claim  volumes,  and  top  five  denial 
reasons.  Two  of  TRICARE  Northwest's  "Border  to  Border" 
objectives  are  targeted  at  improving  claims  processing 
activities. 

Limitations 

As  with  any  research  effort,  limitations  of  the  study  did 
exist .  The  procedures  used  in  the  data  collection  and  content 
analysis  phase  required  some  amount  of  subjective  interpretation 
by  the  resident  and  organizational  points  of  contact.  Attempts 
were  made,  however,  to  reduce  the  influence  of  subjectivity  by 
establishing  decision-making  criteria  and  using  the  quantitative 
JUDGE  Model.  Furthermore,  Regions  other  than  6  and  11  might  be 
able  to  benefit  from  this  GMP  effort,  but  the  recommended 
performance  indicators  are  specific  to  Region  6  and  reflect  the 
perspective  of  TRICARE  Southwest.  Those  Regions  just  beginning 
healthcare  delivery  may  be  slow  in  establishing  the  processes  to 
collect  the  appropriate  data  and  will  also  need  to  measure  other 
metrics  associated  with  a  new  MCSC. 

Discussion 

Clearly,  the  findings  of  this  research  provide  some 
applicable  information  for  Office  of  the  Lead  Agent.  Comparing 
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TRICARE  Southwest  with  TRICARE  Northwest  and  PacifiCare  of  Texas 
yielded  both  similarities  and  differences  that  are  noteworthy. 
Furthermore,  the  key  characteristics  of  the  critical  performance 
indicators  spotlight  the  fundamental  requirements  of  appropriate 
health  plan  metrics.  Finally,  there  are  challenges  that  must  be 
faced  and,  more  importantly,  addressed  before  an  appropriate 
performance  measurement  system  can  be  implemented.  These  issues 
(similarities/differences,  key  traits,  and  challenges)  are 
addressed  in  detail  below. 

Similarities  and  Differences 

As  might  be  expected,  the  similarities  among  the  three 
organizations  were  most  often  between  the  two  DoD  entities.  The 
fact  that  both  Lead  Agent  Offices  work  with  the  same  Managed 
Care  Support  Contractor  results  in  common  CDRLs  in  many 
instances.  Other  similarities  noted  were  the  metrics 
established  by  TRICARE  Southwest  for  the  TRICARE  Senior  Prime 
program  and  the  metrics  used  by  PacifiCare  for  their  senior 
citizen  product  line.  Secure  Horizons.  This  is  largely  due  to 
the  requirements  that  the  Health  Care  Financing  Administration 
(HCFA)  places  on  health  plans  for  Medicare  patients. 

The  differences  among  the  metrics  of  the  three 
organizations  also  highlight  some  interesting  points. 

PacifiCare  of  Texas,  as  a  for-profit  company,  certainly  had  more 
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financial  tneasuiremants  than  the  two  federal  organizations. 

These  measurements  not  only  addressed  cost  accounting  but  also 
revenue  management.  Additionally,  it  should  be  noted  that 
PacifiCare  largely  monitored  the  same  metrics  for  both  their 
commercial  and  senior  product  lines.  Whereas,  TRICARE  Southwest 
segregates  their  existing  performance  measurement  processes. 
Finally,  TRICARE  Northwest  grouped  their  metrics  by  the 
organization's  goals  and  objectives,  while  TRICARE  Southwest 
does  not  have  an  existing  system  in  place  for  indexing  the 
metrics  utilized. 

Key  Traits 

Regardless  of  the  source  organization,  the  critical 
performance  indicators  identified  possess  some  key  traits  that 
contribute  extensively  to  their  effectiveness  as  management 
metrics.  For  those  metrics  that  address  the  health  status  of 
the  population,  an  important  feature  was  a  focus  on  conditions 
that  are  principally  treatable  through  modification  of  health 
behaviors .  This  provides  the  organization  with  targeted 
opportunities  for  health  promotion  and  preventive  medicine 
efforts  with  the  goal  of  improved  patient  outcomes.  Examples 
include  smoking  cessation,  cardiovascular  exercise,  and 
nutritional  education  for  health  plan  membership.  Another  focus 
of  health  status  metrics  is  the  utilization  of  health  services. 
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The  organization  can  use  this  information  to  consider  the 
deployment  of  demand  management  techniques  as  well  as  to 
determine  the  acuity  level  of  their  health  plan  membership. 

The  critical  performance  indicators  for  operations  and 
member  services  also  have  some  distinguishing  features. 
Compliance  with  standards  is  important  for  ensuring  access  to 
care,  and  it  also  impacts  customer  satisfaction  levels.  Member 
services  such  as  toll-free  telephone  lines  must  be  monitored 
rigorously  to  ensure  that  customers  receive  timely,  accurate 
information.  In  addition,  enrollment  processes  must  be  clearly 
defined  and  efficient,  not  only  to  expedite  eligibility 
verification  but  more  importantly  to  provide  health  plan  members 
with  the  tools  needed  to  access  health  services.  Finally, 
provider- focused  metrics  must  address  credent ialing  processes 
and  network  adequacy. 

Cost  accountability  has  become  increasingly  important  for 
Lead  Agent  Offices.  With  the  potential  implementation  of 
Enrollment  Based  Capitation,  the  fiscal  state  of  the  region  must 
be  assessed  proactively  so  that  proper  arrangements  are 
established  with  the  Managed  Care  Support  Contractors.  TRICARE 
Northwest  has  established  metrics  to  accomplish  this,  providing 
true  indicators  of  a  critical  performance  area.  Resource 
sharing  opportunities  must  also  be  identified  and  evaluated  to 
determine  potential  savings.  Another  area  that  has  drawn 
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national  attention  for  TRICARE,  more  so  than  civilian  health 
plans,  is  claims  processing.  Several  critical  performance 
indicators  for  claims  processing  should  be  monitored  regularly 
to  maintain  productive  relationships  with  civilian  network 
providers . 

Challenges  for  Implementation 

Undoubtedly,  there  are  some  formidable  challenges  for 
TRICARE  Southwest  as  they  seek  to  establish  an  effective 

performance  management  system.  Issues  surrounding  data  quality 

/ 

should  be  addressed  regionally,  just  as  they  currently  are  at 
the  DoD  level.  It  is  also  vital  that  the  Lead  Agent  Office  hold 
the  entire  health  system  accountable  and  not  solely  the 
contracted  portion.  Moreover,  for  a  performance  measurement 
system  to  be  implemented,  TRICARE  Southwest's  executive  team 
should  focus  its  sights  on  the  strategic  landscape  they  are 
facing.  This  requires  that  they  identify  the  internal 
organization's  strengths  and  weaknesses  and  the  external 
environment's  opportunities  and  threats. 

From  discussions  with  and  actions  of  MHJS  leadership,  it  is 
clear  that  several  concerns  surround  data  quality .  The  usage  of 
multiple  information  systems,  that  are  often  incompatible/ 
hamper  data  extraction.  Issues  surrounding  the  uncertain 
accuracy  of  the  data  that  is  collected  give  rise  to  speculation 
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among  decisionmakers.  The  lack  of  standard  business  practices 
among  the  three  services  {Army,  Navy,  and  Air  Force)  further 
exacerbates  the  problem.  To  address  these  and  other  data 
cpiality  concerns,  national  MHS  organizations,  such  as  TMA,  are 
creating  teams  focused  on  data  management.  A  continuous 
improvement  approach  will  be  necessary  to  navigate  this 
complicated  terrain,  and  TRICARE  Southwest  should  plan  to 
address  these  hurdles  from  the  regional  perspective. 

A  second  major  challenge  for  the  Lead  Agent  Office  in 
®®^^^^i®hing  performance  measurement  processes  is  the  disparity 
that  exists  between  the  MTFs  and  civilian  provider  networks. 
While  Lead  Agent  staff  members  aggressively  monitor  the  MCS 
Contractors  performance,  MTFs  are  not  held  accountable  for  the 
same  results.  This  is  due  to  the  lack  of  command  and  control 
authority  of  Lead  Agents  over  the  MTFs  in  TRICARE  Regions.  The 
three  services  maintain  authority  over  their  respective 
facilities.  This  presents  a  tremendous  obstacle  for  effective 
and  efficient  operations  from  a  regional  health  plan 
perspective.  Notwithstanding  the  current  MHS  structure.  Lead 
Agent  Offices,  such  as  TRICARE  Southwest,  can  seek  to  identify 
MTF  performance  deficiencies  through  a  systems  approach  and 
through  tactful,  diplomatic  relationships  improve  overall  MHS 
performance . 
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Finally,  TRICARE  Southwest  must  build  upon  clearly  defined 
goals  and  objectives  as  they  address  performance  measurement 
activities.  The  organization  currently  has  three  overarching 
goals:  (1)  optimize  the  health  status  or  the  regional 

population;  (2)  optimize  member- focused  services  for  the 
regional  population;  and  (3)  optimize  fiscal  performance  in  the 
region.  The  critical  performance  indicators  identified  for  each 
of  the  3  core  domains  and  the  aforementioned  related  goals 
should  assist  the  executive  staff  in  determining  measurable 
objectives.  Once  established,  these  foundational  elements 
should  provide  TRICARE  Southwest  with  the  basis  for  an  effective 
performance  measurement  system. 

Conclusions  and  Recommendations 
As  the  Region  6  Lead  Agent  Office  prepares  for  future  MHS 
rightsizing  and  potential  restructuring,  a  well  established 
performance  measurement  system  that  monitors  the  critical 
performance  indicators  will  provide  a  mechanism  for  executive 
management  decision  support.  Through  a  comparative  analysis  of 
similar  health  plans,  the  critical  indicators  have  been 
identified  for  each  of  the  core  domains.  Appendix  H  provides  a 
matrix  that  lists,  by  their  core  domains,  the  critical 
performance  indicators  identified  through  the  JUDGE  Model 
calculations  and  provides  executives  a  tool  to  indicate 
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applicability  across  the  TRICARE  Southwest's  "Family  of  Health 
Plans . " 

This  GMP  wholeheartedly  concurs  with  the  fundamental 
elements  required  for  an  effective  performance  measurement 
process  as  identified  in  1997  by  the  National  Performance 
Review.  The  central  findings  of  the  "Benchmarking  Study  Report" 
were : 

-  Leadership  is  critical  in  designing  and  deploying 
effective  performance  measurement  and  management  systems. 

-  A  conceptual  framework  is  needed  for  the  performance 
measurement  and  management  system. 

Effective  internal  and  external  communications  are  the 
keys  to  successful  performance  measurement . 

-  Accountability  for  results  must  be  clearly  assigned  and 
well  understood. 

*  Performance  measurement  systems  must  provide  intelligence 
for  decisionmakers,  not  just  compile  data. 

-  Compensation,  rewards,  and  recognition  should  be  linked 
to  performance  measurements . 

"  Performance  measurement  systems  should  be  positive,  not 
punitive . 

Results  and  progress  toward  program  commitments  should  be 
openly  shared  with  employees,  customers,  and 


stakeholders . 
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The  executive  staff  of  TRICARE  Southwest  should  consider  these 
tenets  of  quality  management  as  they  further  develop  their 
performance  measurement  processes. 

Without  a  doubt,  the  MHS  will  continue  to  face  budgetary 
pressures.  Identifying  the  critical  performance  indicators  for 
regional  management  of  TRICARE  is  simply  the  beginning  of  the 
continuous  improvement  efforts  that  should  be  sought.  Future 
research  should  be  targeted  toward  improving  the  overall 
operations  of  the  TRICARE  program  through  trend  analyses  and 
business  practice  enhancements.  Furthermore,  additional  studies 
to  determine  specific  benchmarking  opportunities  with  civilian 
health  plans  would  benefit  the  MHS  as  it  seeks  to  implement  the 
industry' s  best  practices . 

The  Lead  Agent  Office  of  Region  6  is  well  positioned  for  a 
dynamic  future ^  In  their  quest  to  optimize  MHS  performance,  the 
established  core  domains  and  goals  provide  targets  to  guide  the 
arrows  of  effort .  A  comparison  with  similar  health  plans 
allowed  the  identification  of  critical  performance  indicators  to 
be  monitored.  Through  consistent  assessment  activities,  TRICARE 
Southwest  can  translate  vision  into  reality  and  truly  become  the 


premier  MHS  health  plan. 
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Operations /Member  Services 
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Disenrollment  Survey  Statistics 


Criticat  Parfoimanca  Indicator  Candidates 


Critical  Performance  Indicator  Candidates 


Page  7  of  7 


Appendix  F 
Cost  Accountability 
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Appendix  G 
JUDGE  Models 


Judging  Utility:  a  Decision  Generator  and  Evaluator 
Determining  Appropriate  Performance  Indicators  Through  a  Comparative  Analysis  of  Similar  Health  Plans 
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Critical  Performance  Indicators 
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TSW _ Cl  Health  Risk  Appraisal  (HEAR  Stats) 

PoT _ Cl  Diabetes _ 

PoT  Cl  Women’s  and  children’s  health 

PoT  Cl  Cardiovascular  health  _ 
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Beneficiaries  telephoning  the  TSC  shall  be 
TSW _ CS  placed  on  hold  for  no  longer  than  5  minutes. 
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TNW _ EN  Reason  for  Disenrollment 

TNW  EN  Disenrollment  Survey  Statistics 
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Resource  Sharing  -  Estimated  Target  Savings 

TNW _ RS  vs  Actual  Savings  (Aggregate  &  by  MTF) 

TNW _ RS  Resource  Sharing  -  MTF  Agreement  Status 
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